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This study investigates the frequency of research-based leadership strategies utilized by 
newly hired school principals in the workplace. Public school superintendents in Indiana 
were asked to respond to two open-ended research questions. Through the use of content 
analysis, their comments were coded for the occurrence of effective leadership practices. 
The Educational Leadership Constituent Council (ELCC) standards were used as 
classification categories. The findings revealed that collaboration and skills in 
instruction, curriculum, and evaluation were the most frequently observed leadership 
skills. Management skills were identified as the area in greatest need of improvement, 
especially a noted lack of budgetary skills. The content analysis identified categories of 
responses in addition to the ELCC standards. Superintendents repeatedly commented on 
new principals ’ strong interpersonal skills and suggested that additional years of 
experience would enhance the principals ’ development of the most influential leadership 
skills. 
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In the last 50 years, researchers have provided ample evidence that effective school 
principals have a positive influence on student achievement and overall school success 
(Branch, Hanushek, & Rivkin, 2013; Leithwood, Seashore, Anderson, & Wahlstrom, 
2004; Rice, 2010). Numerous studies have led to the identification of various leadership 
traits or behaviors that have been documented as contributing to school improvement 
(Marzano, Waters, & McNulty, 2005). Recognizing the importance of this research, the 
purpose of this study was to determine if Indiana’s newly hired principals exhibited these 
effectiveness skills in their assigned schools. This assessment is based on the 
observations and opinions of the superintendent and centered on the Educational 
Leadership Constituent Council Standards (2011) for building-level leaders. In Indiana, a 
Master’s degree comprised of a standards-based university administrative preparatory 
program, is required in order to receive a principal’s license. This license is necessary for 
employment as a public school principal in the state (Indiana Department of Education, 
2013). 

Researchers recommend investigating the perfonnance of new principals in 
relationship to shared standards for leadership preparation programs (Andenoro et al., 
2013). The study that follows is a companion study to one completed by Boyland, 
Lehman, and Sriver (in press), which used quantitative data to assess the standards-based 
proficiency of newly hired principals. This study uses content analysis to further 
examine the behaviors of newly hired principals within the framework of the Educational 
Leadership Constituent Council (ELCC) Standards and research-based principles of 
effective leadership. The ELCC Standards focus on the following areas of leadership: 
vision, instructional program, management, collaboration, ethics, and political, social, 
economic, legal, and cultural contexts. 

Background 

In 1966 the US Department of Education commissioned James Coleman and 
several other scholars to conduct research on the topic of educational equality in the 
United States. A fundamental premise emerging from this study suggested that the 
primary determinants of academic success are a student’s background and socioeconomic 
status. Coleman et al. wrote, “It is known that socioeconomic factors bear a strong 
relationship to academic achievement. When these factors are statistically controlled, 
however, it appears that differences between schools account for only a small fraction of 
differences in pupil achievement” (p. 21). 

Soon thereafter, bureaucrats, educators, and researchers started questioning these 
findings in terms of the school’s influence on student achievement and began subjecting 
the results of Coleman’s study to intense scrutiny (Cain & Watts, 1970; Moynihan, 
1968). The seeds of the effective schools refonn era were sown in this reaction by 
initiating a search for the key elements of successful schools and ultimately validating the 
importance of specific school practices and the fundamental role of the principal. 

Weber (1971) conducted one of the earliest studies designed to detennine the 
characteristics of an effective school. He focused on reading programs in inner city 
schools and cited strong building leadership as one element of a successful school. 
Berman and McLaughlin (1978) found that educational innovations were more successful 
when supported by the building principal. In Brookover and Lezotte’s (1979) analysis of 
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schools with improving student achievement, principals in improving schools were cited 
for their assertiveness in the role of instructional leader. As the decade concluded, 
Edmonds (1979) offered this compelling argument for the importance of the building 
principal: 

I want to end this discussion by noting as unequivocally as I can what seem to 
me the most tangible and indispensable characteristics of effective schools (a) 
they have strong administrative leadership without which the disparate elements 
of good schools can neither be brought together nor kept together,. . . (p. 22). 

During a presentation in 1983, Finn declared, “First, schools make a difference in 
how much children learn. Second, principals make a difference in how effective schools 
are” (p. 3). In 1987, Cawelti wrote, “Research of effective schools has validated the vital 
role of principals in schools that consistently achieve above expectations” (p. 1). He 
identified four behavioral patterns of leaders: vision, organization developer, instructional 
support, and monitoring learning, deemed to be essential in improving school 
productivity. 

Milstein, Bobroff, and Restine (1991) emphasized the importance of the principal 
in school improvement, reporting that successful school refonn requires, “... site-based 
administrators who have vision, beliefs, abilities, and energy required to lead others 
toward shared objectives” (p. 2). In synthesizing research on principal effectiveness, 
Hallinger and Heck (1998) agreed, concluding: “Schools that make a difference in 
student’s learning are led by principals who make a significant and measurable 
contribution to the effectiveness of staff and in the learning of pupils in their charge” (p. 
158). 

As research efforts continued into the next decade, multiple studies contributed to 
the growing body of evidence linking school leadership with student achievement (Cowie 
& Crawford, 2007; Duke, Grogan, Tucker, & Heinecke, 2003; Tucker, Henig, & 
Salmonowicz, 2005). In 2004, Leithwood, Seashore, Anderson, and Wahlstrom reported 
that school leadership was second only to classroom instruction among the school-related 
factors that contributed to student learning. Marzano, Waters, and McNulty (2005) 
conducted a meta-analysis of 35 years of educational research and found a statistically 
significant relationship between principal effectiveness and student achievement, stating 
that, “... school leadership has a substantial effect on student achievement” (p. 12). A 
recent study conducted in Texas found that a highly effective principal raised the 
achievement of a typical student between two and seven months (Branch et ah, 2013). 
Several other recent reports have substantiated the importance of principal effectiveness 
in regard to student achievement (Horng & Leob, 2010; The Wallace Foundation, 2013). 

Guided by the belief that principals do make a difference, educational researchers 
turned their attention to attempting to determine which behaviors are most influential in 
the quest for school improvement. Acknowledging that there is a lack of consistency in 
the terminology used to describe effective behaviors, a typical list of effective leadership 
strategies includes: creating a vision, possessing integrity, knowing oneself, sharing 
success, developing leadership in others, utilizing effective problem solving skills, and 
understanding the organization and the forces that shape it (Bennis, 1989; Carnegie, 
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1936; Collins, 2001; Covey, 2004; Maxwell, 1998). An analysis of noteworthy studies in 
educational leadership yields a comparable list of effective leadership behaviors with 
some important additions. Specifically, effective school leaders must also be adept at 
creating a climate to promote learning; improving, monitoring, and evaluating 
instruction; demonstrating expertise in curriculum and assessment; making data-based 
decisions, and fostering community relationships (Langley & Jacobs, 2006; Leithwood et 
ah, 2004; Marzano et ah, 2005; National Association of Elementary School Principals, 
2013; National Association of Secondary School Principals, 2013; The Wallace 
Foundation, 2013). 

The challenge in identifying cause and effect relationships between specific 
elements of principal leadership and student achievement may be due to the complexity 
of the role. The value of a principal can manifest itself in multiple ways, both directly 
and indirectly. From their review of the research, Homg, Kalogrides, and Loeb (2009) 
proposed that a variety of school outcomes, may be influenced by the effectiveness of a 
principal who recruits and motivates quality teachers, identifies and articulates a school 
vision and goals, allocates resources efficiently, and develops instructional support 
structures. Rice (2010) found that the behaviors of skillful principals influenced several 
areas, including teacher satisfaction and parents’ perceptions about the school, with the 
combined results contributing to improved student academic performance. Hallinger 
(2003) concluded that the importance of effective principal leadership in contributing to 
successful change was a consistent finding in the research on school improvement. 

Without doubt schools are complex and dynamic organizations. The variables 
that influence the success of schools are numerous. In addition to effective school 
leadership, Shannon and Bylsma (2007) list clear and shared vision, high expectations for 
all students, collaboration and communication, alignment with state standards, frequently 
monitored learning and teaching, focused professional development, supportive learning 
environment, and a high level of community involvement as characteristics of a high 
performing school. Each variable contributes to, or detracts from, the effectiveness of the 
school. Multiple studies grounded in effective schools research have identified the 
building principal as a key variable in the operation of an effective school. Today, we 
can state with reasonable assurance that the perfonnance of the building principal greatly 
influences student achievement and other variables that contribute to the success of a 
school. 


Standards-Based Educator Preparation 

There are several organizations and multiple assessment strategies that have served to 
create a baseline of standards and expectations for educator preparation. One such 
organization, the National Council for Accreditation of Teacher Education (NCATE) was 
founded in 1954 and has been pivotal in providing research-based benchmarks for quality 
teacher and administrator preparation programs in the United States. In 2013, NCATE 
and TEAC (Teacher Education Accreditation Council) merged to become CAEP 
(Council for the Accreditation of Educator Preparation). Nationwide, there are over 650 
accredited institutions participating in CAEP as one measure of assurance that educator 
training programs are of satisfactory relevance and quality (CAEP, 2013). 
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High quality preparatory programs are grounded in the implementation of 
pertinent and rigorous standards. For school leadership training programs, the 
Educational Leadership Constituent Council (ELCC, 2011) or the Interstate School 
Leaders Licensure Consortium (ISLLC, 2008) are the national standards commonly 
adopted (CCSSO, 2012; ELCC, 2011). Of the four major Indiana universities offering 
principal training, all follow ELCC Standards, which requires a rigorous curriculum with 
a clinical internship under the supervision of a university supervisor and onsite mentor. 
ELCC Standards are considered Indiana’s primary standards for principal preparation, 
serving accreditation purposes and also guiding best practice. 

Theoretical Framework 

There is a growing body of empirical evidence supporting the link between principal 
effectiveness and student achievement, which provides the theoretical framework for this 
study (Leithwood et ah, 2004; Marzano et al., 2005; Rice, 2010). In Indiana, the ELCC 
Standards are the formally adopted standards by the major universities for principal 
preparation programs. Research shows that certain skills, knowledge, and behaviors, as 
outlined in the ELCC Standards, are known to increase student achievement (Young & 
Mawhinney, 2012). However, in Indiana, there is a research deficit in the area of new 
principals, meaning that no recent studies could be found examining the perfonnance of 
principals in their first, second, and third years in the position. Our theoretical 
framework, which links principal effectiveness with student achievement, prompts the 
question; do recently hired school principals demonstrate the skills, behaviors, and 
knowledge as outlined in the ELCC Standards? Currently, very little is known about the 
performance of Indiana’s new school leaders after they finish their required university 
preparation, providing the catalyst for this study. 

Purpose 

The ELCC Standards provide a research-based framework for university school 
leadership preparation programs. Empirical research thoroughly and consistently 
supports the ELCC Standards as foundational in developing leadership effectiveness, as 
documented by noted educational leaders and scholars like Michelle Young, Hanne 
Mawhinney, Dianne Taylor, Margaret Orr, Diana Pounder, Gary Crow, and Pamela 
Tucker (as cited in Young & Mawhinney, 2012). However, in Indiana, although 
candidates are being prepared using the ELCC Standards, there is a lack of follow-up on 
candidates to determine their effectiveness once they are hired as principals. Researchers 
recommend exploration of shared standards for preparation programs in order to develop 
a deeper understanding of leadership education outcomes (Andenoro et al., 2013). 
Accordingly, our purpose was to assess the effectiveness of recently hired principals by 
comparing their performance with the ELCC Standards. Our investigation sought to 
detennine if newly hired principals, those in their first, second or third year, demonstrated 
the knowledge, skills, and behaviors identified as necessary contributors to school 
improvement. 
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Methodology 


Survey methodology was deemed most appropriate for exploring our research questions. 
Surveys are recommended as an economical and efficient method of collecting data from 
a large sample (Scholls & Smith, 1999). We sought feedback directly from Indiana’s 289 
public school superintendents because these are the individuals responsible for the hiring 
and evaluation of principals. 

Survey Instrument and Participant Selection 

A three-part electronic survey was developed for use in this study. The research team 
developed the survey instrument using the ELCC Standards and specific questions 
regarding new principals’ strengths or areas for improvement, which were derived from 
our research questions. Prior to gathering data, a six-member panel consisting of 
university faculty, superintendents, and principals with experience in survey development 
vetted the instrument and submitted feedback regarding face and content validity. Based 
on the panel’s feedback, several revisions were made to wording and sequencing, which 
improved the survey’s clarity and focus. The survey was then pilot tested using a similar 
panel consisting of former school superintendents, principals, and university faculty 
members who suggested no additional revisions. The survey was administered using the 
platform Qualtrics (www.qualtrics.com) and emailed to each of the 289 Indiana 
superintendents in the spring of 2013. 

The first section of the survey gathered basic demographic information about the 
school to which the new principal was assigned including the school’s grade levels, 
population characteristics, and type of school community (rural, suburban, or urban). In 
the second section of the survey, superintendents were asked to rate the effectiveness of 
the new principal using the six categories of the ELCC Standards as assessment criteria. 
In this section of the survey there were 33 forced choice items designed to gather the 
superintendent’s perceptions about the new principal’s effectiveness. 

The third section of the survey served as the source of data for this study. In this 
section of the survey, superintendents were asked the two following research questions: 

1. What strengths does this newly hired principal possess that makes him/her a 
highly effective building leader? 

2. In what areas does this newly hired principal need to improve his/her level of 
effectiveness? 

Following Institutional Review Board approval, all 289 public school superintendents 
in Indiana were invited to participate in the study. In May of 2013, an introductory email 
with the survey link was sent to all superintendents, asking them to complete a survey for 
each principal hired who had completed a university leadership preparation program 
since 2009. The survey responses were anonymous and no identifying information was 
requested. From a total of 289, 53 usable surveys were returned, yielding a response rate 
of 17%, which is considered acceptable for electronic surveys (Sheehan, 2001). The 
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survey responses were anonymous and no identifying information was requested. If 
identifying information was voluntarily provided it was not maintained. 


Data Analysis 

A content analysis of the responses to the two research questions listed above was 
conducted following the principles outlined by Guba and Lincoln (1981) and Holsti 
(1969). The general research process is defined as, “Content analysis is any technique for 
making inferences by objectively and systematically identifying specified characteristics 
of messages” (Holsti, 1969, p. 14). 

Coding data is a key function in content analysis research. In this study, the 
ELCC Standards were used as the content analysis categories because they afforded 
alignment with the research questions and a functional and systematic focus for 
investigation. Since most responses were in the fonn of one or several sentences, a single 
word or theme was selected as the recording unit. Coding reliability was achieved by 
using multiple reviewers and applying the Kappa Statistic to measure interrater 
reliability. The team of reviewers met to decide upon the following coding rules: 
definition of research problem in terms of categories, coding unit, and coding 
enumeration (Holsti, 1969). After agreement was reached on the coding requirements, 
each reviewer worked independently, and used the same printed set of the ELCC 
Standards for reference. The results of the coding process were submitted to the authors 
for compilation. 

Since more than one reviewer was utilized in this research, there was a need to 
assess interrater reliability. The kappa statistic was selected as the measure of reliability 
(See Table 1). 

Table 1 

Interpretation of Kappa _ 

Almost 

Poor Slight Fair Moderate Substantial Perfect 


0.00 0.20 0.40 0.60 0.80 1.00 

Agreement 

Less than chance agreement 
Slight agreement 
Fair agreement 
Moderate agreement 
Substantial Agreement 
Almost perfect agreement 


Kappa 

Kappa 

<0 

0 . 01 - 0.20 

0.21-0.40 

0.41-0.60 

0.61-0.80 

0.81-0.99 
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The importance of this statistic is that it represents how frequently the data 
analyzed by the coders are assigned to the same category. According to Carletta (1996), 
Kappa is widely accepted, is interpretable, and allows for different results to be 
compared. It is typically used to assess the degree to which two or more raters agree 
when assigning data to categories. Kappa provides a numerical rating of the degree to 
which observers agree when evaluating the same item. There are various scales used to 
describe the level of agreement for kappa values. The one used in this study is derived 
from Viera and Garrett (2005). 

Results 

In the spring of 2013, Indiana public school superintendents were asked to 
complete a survey on the effectiveness of each new school principal hired in their 
districts. There were a total of 53 usable surveys returned for analysis; however, 15 of 
the new administrators were assigned to positions at the district level. Our focus for this 
report was specifically building-level leaders. Therefore, we will be reporting on the 37 
responses from superintendents regarding new building-level administrators. Of these 37 
building-level leaders, 51.4% (n = 19) were identified as principals, 43.2% (n = 16) as 
assistant principals, and 5.4% (n = 2) as “other building-level administrative” positions. 
Because 35 of the 37 building-level leaders were assistant principals or principals, we 
refer to the group as “principals.” All of the principals whose perfonnance is reported on 
were in their first, second, or third years in the positions. 

Demographic Profile of Principals’ Schools 

Superintendents were asked to provide demographic information about the schools where 
the new principals were assigned. The grade levels of the principals’ schools were 
evenly divided with 48.6% (n = 18) being elementary or intennediate-level schools, and 
48.6% (n = 18) being middle or high schools. There was one school (2.7%) that housed 
all grades levels, K-12. Most of the schools, 91.4% (n = 34) enrolled 20% or less 
minority students, with 5.4% (n = 2) enrolling 21 - 40% minority students, and one 
school (2.7%) enrolling 61 - 80% minority students. The percentage of students 
qualifying for free or reduced meals indicated that 5.4% (n = 2) of the schools had less 
than 20% qualifying, 51.4% (n = 19) had 21 - 40% qualifying, 32.4% (n = 12) had 41 - 
60% of students qualifying, and 10.8% (n = 4) had 61% or above qualifying for meal 
assistance. The last demographic question was on community type, revealing that the 
majority of the schools were in rural areas at 78.4% (n = 29), with suburban at 16.2% (n 
= 6), and urban at 5.4% (n = 2). In general, the demographics of the schools and 
communities presented an accurate representation of Indiana in terms of typical 
population distributions and characteristics (U.S. Census Bureau, 2013). For the most 
part, Indiana is a rural state, with only 16 areas classified as “large urban” (Indiana State 
Government, 2009). 
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Analysis of Responses to Open Ended Questions - Standards 

In response to the first research question, which asked superintendents to cite the ELCC 
standard areas in which newly hired principals were more effective, all six categories 
were mentioned (See Table 2). 

Table 2 


Frequency of superintendents’ comments noting principals' strengths 


Std 

Description 

n 

% 

Kappa 

z 

P 

1 

Vision 

20 

9.3% 

0.36 

1.87 

0.061 

2 

Culture and Inst Program 

64 

29.9% 

0.72 

8.42 

0.000 

3 

Management 

41 

19.2% 

0.32 

2.76 

0.006 

4 

Collaboration 

67 

31.3% 

0.67 

7.79 

0.000 

5 

Integrity, Fairness, and Ethics 

15 

7.0% 

-0.04 

-0.18 

0.857 

6 

Pol, Soc, Econ, Legal & Cul 

7 

3.3% 

0.23 

0.47 

0.638 


Note, n = number of comments received from superintendents 


Standard four, the education leader applies knowledge that promotes the success of every 
student by collaborating with faculty and community members, was the most frequently 
cited (n = 67, % = 31.3). Examples of comments typical of this category include, “He 
creates a very positive climate for students and parents,” and “. . . ability to work with 
teachers on their level, relates well to the community, trusts employees to do the jobs she 
gives them.” The kappa statistic for this standard is 0.67 which classifies the interrater 
reliability as having substantial agreement. 

Behaviors aligned with standard two, the education leader applies knowledge that 
promotes the success of every student by sustaining a school culture and instructional 
program conducive to student learning, was the second most often cited (n = 64, % = 
29.9) category. Typical remarks associated with this category include, “Ability to 
analyze data, ability to identify appropriate strategies, create a culture of high 
expectations, strong disciplinarian,” and “knowledgeable in instructional leadership and 
evaluation.” The kappa statistic for this standard is 0.72 which classifies the interrater 
reliability as having substantial agreement. 

Standard three, the education leader promotes the success of every student 
through monitoring and evaluating the school management and operational systems, was 
mentioned in nearly one-fifth (n = 42, % = 19.2) of the responses. Superintendents 
routinely described these behaviors in the following manner, “project management, 
logistics” and “operations skills.” The kappa statistic for this standard is 0.32 which 
classifies the interrater reliability as having fair agreement. 
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Of the remaining three standards, one, five, and six, all were mentioned less than 
10% of the time. This of course is not meant to suggest that the newly hired principals do 
not possess these traits, but simply that their superintendents may have observed other 
behaviors more frequently. The kappa statistics for standards one (0.36) and six (0.23) 
suggested fair agreement among the coders. There was less than chance agreement for 
standard five. 

The second research question asked superintendents to suggest areas in which the 
newly hired principals needed to improve their level of effectiveness. Similar to the 
responses to the identification of strengths, areas for improvement included all six 
standards categories. By far the most frequently cited area in need of improvement was 
related to standard three (n = 49, % = 30.6), management (See Table 3). 

Table 3 


Frequency of superintendents' comments noting areas for principals ’ improvement 


Std 

Description 

n 

% 

Kappa 

z 

P 

1 

Vision 

18 

11.3% 

0.36 

1.87 

0.061 

2 

Culture and Inst Program 

31 

19.4% 

0.72 

8.42 

0.000 

3 

Management 

49 

30.6% 

0.32 

2.76 

0.006 

4 

Collaboration 

32 

20.0% 

0.67 

7.79 

0.000 

5 

Integrity, Fairness, and Ethics 

4 

2.5% 

-0.04 

-0.18 

0.857 

6 

Pol, Soc, Econ, Legal & Cul 

26 

16.3% 

0.23 

0.47 

0.638 


Note, n = number of comments received from superintendents 


Over thirty percent of the superintendents commented that newly hired principals needed 
to improve in this area. Of the 49 coded responses under the heading of standard three, 
28.6% described the newly hired principal as needing to improve in time management. 
Nearly one-fourth, 24.5%, of the respondents identified limited skills in school finance 
and budgets among new hires. Approximately one-fifth, 20.4%, of the recently employed 
principals needed to improve their communication skills. The kappa statistic for this 
standard is 0.32 which classifies the interrater reliability as having fair agreement. 

A nearly equal number of suggestions to improve leadership behaviors classified 
under standards two (n = 31, % = 19.4) and four (n = 32, % = 20%) comprised the second 
tier of areas for improvement. With regard to standard two, superintendents routinely 
remarked about the principal’s inability to effectively evaluate teachers, a lack of 
recognition of good teaching, and a lack of familiarity with academic standards. The 
kappa statistic for this standard is 0.72 which classifies the interrater reliability as having 
substantial agreement. 

The need to improve the skills described in standard six (n = 26, % = 16.3), an 
education leader promotes the success of every student by evaluating the potential moral 
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and legal consequences of decision making in the school, merits a comment in this 
discussion. Over 25% of the statements noting the need to improve were related to an 
understanding of the law. Slightly less than 25% of the suggestions for improvement 
centered on the need to acquire a better understanding of the political environments that 
influence schools. The kappa statistic for this standard is 0.23 which classifies the 
interrater reliability as having fair agreement. 

Analysis of Responses to Open Ended Questions - Other Performance Measures 

During the content analysis it became apparent that a number of comments were 
not aligned with the six ELCC Standards but still merited consideration. To 
accommodate a review of these data, the coders collaborated on the development of other 
categories that appeared in the analysis then followed the content analysis protocols 
described above to further derive frequently cited suggestions from survey respondents. 
This process yielded four categories: interpersonal skills, personal traits/attitude, 
experience, and miscellaneous. To be consistent with the data classified according to the 
standards, the other categories were divided into areas of strength and those in need of 
improvement. 

Superintendents listed interpersonal skills as a strength in more than 50% (n = 29, 
% = 55.9) of the responses assigned to this category. Descriptive phrases such as, “has a 
positive attitude and personality that connects with kids, teachers, and parents,” “people 
skills,” and “excellent judgment,” were routinely found in written responses. The kappa 
statistic for this standard is 0.36 which classifies the interrater reliability as having fair 
agreement. Closely associated with interpersonal skills is the second most often cited 
catergory, personal traits/skills. The distinction was made because these comments 
depicted a different attribute. Of the 13 comments assigned to this category, 61.5% 
complimented the new hire as “dedicated” or having a strong work ethic. Being a good 
listener or having an eagerness to learn were other infonnative remarks. The kappa 
statistic for this category is 0.72 which classifies the interrater reliability as having 
substantial agreement. The data reported as other measures of the strengths of newly 
hired principals appear below in Table 4. 

Table 4 


Frequency of superintendents’ comments noting principals' strengths 


# 

Description 

n 

% 

Kappa 

z 

P 

1 

Interpersonal Skills 

29 

55.9% 

0.36 

1.87 

0.061 

2 

Personal Traits/Attitude 

13 

25.0% 

0.72 

8.42 

0.000 

3 

Experience 

0 

0.0% 

0.32 

2.76 

0.006 

4 

Miscellaneous 

10 

19.2% 

0.67 

7.79 

0.000 


Note, n = number of comments received from superintendents 
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Other Performance Measures 


In the categories listed as other measures, areas of needed principal improvement, there is 
only one item of note. From the comments submitted by the superintendents, experience 
will be the key to more effective performance. Repeatedly the phase, “more experience” 
was cited in response to the research question. 

In this context, more experience referred to longevity in the role as opposed to a 
broader range of tasks or responsibilities on the job. There were some instances in which 
this term was associated with “confidence,” but the message of gaining experience was 
clear in an overwhelming number (n = 25, % = 64.1) of responses. The kappa statistic 
for this category is 0.32 which classifies the interrater reliability as having fair agreement 
(See Table 5). 

Table 5 


Frequency of superintendents’ comments noting areas for principals’ improvement 


# 

Description 

n 

% 

Kappa 

z 

P 

1 

Interpersonal Skills 

3 

7.7% 

0.36 

1.87 

0.061 

2 

Personal Traits/Attitude 

4 

10.3% 

0.72 

8.42 

0.000 

3 

Experience 

25 

64.1% 

0.32 

2.76 

0.006 

4 

Miscellaneous 

7 

17.9% 

0.67 

7.79 

0.000 


Note, n = number of comments received from superintendents 


Summary and Discussion 

The overarching goal of this study was to determine if newly hired principals are 
demonstrating the skills and behaviors research has identified as necessary ingredients for 
school improvement. A synthesis of these skills and behaviors typically includes: 
creating a vision, developing leadership in others, utilizing effective problem-solving 
strategies, promoting a climate for learning, evaluating and improving instruction, 
making data-based decisions, and forging strong community relationships. These factors 
are closely aligned with the ELCC Standards, which guide administrator preparation 
programs in over 650 universities throughout the nation. The findings contained in this 
study are equally important for university administrator preparation programs to prompt 
an assessment of curriculum and training strategies. There is evidence from this study 
that effective leadership practices are being utilized by newly hired principals. 

A summary of the results depicts a contrast of the leadership traits and practices 
public school superintendents identified as effective and those in need of improvement 
while observing the perfonnance of newly hired principals. These data provide evidence 
that all categories of standards-based leadership practices were observed under job 
conditions. It is also apparent that these practices are being employed with varying 
frequencies. The data document areas of strength with regard to the utilization of 
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effective leadership practices. The data also suggest areas where the use of these 
practices should be improved. A summary of the data is illustrated in Figure 1. 


Frequency of ELCC Standards-based Leadership Practices Identified 

by Content Analysis 


Std 1 Vision 
Std 2 Culture and Inst Prog 
Std 3 Management 
Std 4 Collaboration 
Std 5 Integ, Fairness, Ethics 
Std 6 Pol, Soc, Econ, Legal, Cul 

0 10 20 30 40 50 60 

□ Imp. Needed ■ Strength 

Figure 1 

One of the more encouraging results is that principals were more frequently cited 
for demonstrating practices associated with improving student achievement. Specifically, 
utilizing leadership strategies related to standards two and four were identified more often 
than the remaining standards. Both categories are aligned with strategies known to 
positively influence school perfonnance; monitoring and evaluating instruction, and 
fostering community relationships (Clifford, Behrstock, & Fetters, 2012; National 
Association of Secondary School Principals, 2013; The Wallace Foundation, 2013). 
These findings are related to the first research question. 

Behaviors classified under standard four were identified as strengths of newly 
hired principals more than any other trait (n = 67). This standard is described as 
promoting the success of students by collaborating with faculty and community members. 
There were also some suggestions for improvement under this heading (n = 32) but the 
descriptions of collaboration as a strength more than doubled the number of remarks 
about needing improvement. The second most often (n = 64) mentioned category 
described behaviors grouped under standard two. This standard is characterized by the 
development of a rigorous curricular program and supervision of instruction. Comments 
identified as describing this standard as a strength of newly hired principals also doubled 
the number of observations indicating a need for improvement. It is important to note 
that for the content analysis of these factors, the interrater reliability kappa statistic was in 
the substantial agreement range for all four classifications. 

Responses depicting skills related to standard three, monitoring and evaluating the 
school management and operational systems, routinely (n = 41) identified these behaviors 
as an area of strength. Comments such as having organizational and management skills 
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and being detail oriented were typical descriptive phrases. However, the need to improve 
management skills received the greatest number of needs improvement comments (n = 
49) of any standard. This difference of nearly 20% is due primarily to the 
superintendents’ assessment that newly hired principals need to improve in the areas of 
budging and school finance. The interrater reliability range for both measures was in the 
fair agreement range. 

The content analysis identified a nearly equal number of classifications as 
strengths (n = 20) and needs improvement (n = 18) under standard one, vision. Some 
principals were viewed as visionary; others were described as being limited in their 
ability to effectively plan for school improvement. The kappa statistic for these measures 
placed the interrater reliability in the fair agreement range. 

Regarding standard five, which deals with fairness and integrity of administrative 
actions, the acknowledgement of this characteristic as a strength surpassed it being a 
weakness by a count of nearly four to one. It is important to note here that just because 
the concept of fairness and integrity was less frequently cited overall by superintendents, 
it should not be viewed as less evident or unimportant. Our companion study found that 
newly hired principals bordered on perfonning at a distinguished level when ethical 
behavior was considered (Boyland et ah, in press). The need to improve in the political, 
social, economic, legal, and cultural areas yielded the highest percentage disparity when 
compared to contexts in which these behaviors were viewed as strengths. Limitations in 
this area centered on a lack of proficiency in the political arena and in dealing with legal 
matters. 

A review of the other categories identified through content analysis generated two 
noteworthy factors. In the distillation of other strengths, twenty-nine comments, almost 
three times as many as the second most mentioned category, were recorded as 
interpersonal skills. The notion of being able to develop positive working relationships 
with members of the faculty, community, and student body remains an integral factor in 
school leadership (Langley & Jacobs, 2006). 

The analysis of the need for improvement categories under the other heading also 
identified one dominant response. Nearly 65% of the comments suggested that newly 
hired administrators would improve with experience. This is a logical conclusion but 
prompts a number of important considerations, including two key questions. First, can 
newly hired principals be better prepared so that the progression of skills from novice to 
proficient can be accelerated? Second, can we count on the mentors currently in the field 
to provide proper guidance? 

Implications and Recommendations for Practice 

Research clearly documents that effective school principals play an important role in 
improving student achievement. At the same time, there are criticisms of university 
preparatory programs responsible for training principals. For example, Cowie and 
Crawford (2007) called principal preparation programs an “act of faith” (p. 129). Levine, 
a strong critic of university administrator preparation programs, referred to them as “... 
the weakest of all the programs at the nation’s education schools” (2005, p. 13). Hess 
and Kelly stated, “Because preparation of principals has not kept pace with changes in 


104 



the larger world of schooling, graduates of principal preparation programs have been left 
ill equipped for the challenges and opportunities posed by an era of accountability” 
(2005, p. 40). These and other reports challenging the quality and relevance of university 
principal preparation programs raise questions regarding how well new principals 
function once they are on the job. 

Consequently, the outcomes of this study create implications for practice and 
research because our results document standards-based areas in which new principals, per 
their superintendents, were perceived as effective. Our results suggest that university 
preparatory programs in Indiana, at least to the extent measured by our instrument and 
per the ELCC Standards, are preparing candidates for the real-world of leadership as seen 
through the eyes of their superintendents. In addition, many of the areas that 
superintendents reported as strengths for new principals, for example, collaboration with 
faculty and the community, supervision of instruction, and development of rigorous 
programs, are directly related to areas necessary in establishing conditions for improving 
student achievement. 


Limitations 

This study has several limitations. One limitation is that the survey response rate was 
only 17% of the superintendents in the state. Although this is considered acceptable for 
electronic survey research, it limits the generalizability of results. Therefore, the reader is 
advised to view these results as exploratory. 

Another limitation is that the survey was conducted only in Indiana. It seems 
logical that there is some similarity throughout the nation in the challenges facing new 
administrators in their first administrative assignment. Certainly other authors have 
enumerated these challenges from a universal perspective, but this study does not 
presently contain evidence to extend this assumption beyond state boundaries. 

Need for Further Research 

The results of this study were encouraging because superintendents largely reported that 
newly hired principals were demonstrating behaviors and skills aligned with effective 
leadership practices (Kaplan, Owings, & Nunnery, 2005; Marzano et al., 2005). 
Nevertheless, there is a call for further research. The goal of using national standards in 
the development of administrator preparation programs is to create a framework for 
designing curricula better aligned to the challenges a novice administrator will face on 
the job (Hambrick-Hill, Tucker, & Young, 2012). University preparation programs 
should explore means by which knowledge and skills acquired in the classroom can be 
more effectively transferred to the workplace (Barnett, 2005). To accomplish these 
goals, studies designed to assess the effectiveness of new principals should directly 
capture their voices, needs, and opinions regarding their own perfonnance and 
preparation. These data should then be used for programmatic and curricular planning at 
the university level, and also to provide supportive assistance and resources. 

In addition, further studies encompassing wider geographic areas and using larger 
sample sizes are necessary in order to better understand the preparatory needs of new 
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school leaders and their additional needs for professional development and support once 
they become principals, as well as to monitor their effectiveness after they have been on 
the job for several years. Since a conspicuous number of superintendents’ remarked that 
additional years of experience was what newly hired principals needed to improve, it 
would be interesting to test this assumption. 

The induction of the novice administrator into the profession is also in need of 
further study. The chance meeting with a superintendent or the routinely scheduled 
districtwide administrators’ meeting is not sufficient for the professional development 
required for today’s principals to make a difference in the lives of the students being 
served. According to Kearney (2010) induction programs should be standards-based, 
including coaching, and collect data to document the effectiveness of the newly hired 
principal. Each of these strategies has the potential to improve the likelihood that novice 
principals will more readily demonstrate effective leadership behaviors. 

In summary, this study provided evidence that effective leadership practices are 
being utilized in Indiana. The ongoing question is, of course, can these strategies become 
pervasively employed in all schools by every school leader? 
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